Robust Movement Policies for Physics-based Characters in 2-D

This course project outlines an approach for constructing simple, relatively robust control strategies for a simple, physics-based biped character. A simple, articulated character was first built from links in the BOX2D simulator, and tested during each phase of creation. These phases consisted of specifying torques for each joint of the character, creating Proportional Derivative (PD) controllers to actuate or move these links about their joints, defining Finite State Machine (FSM) biped locomotion movement routine controllers and a basic balance strategy for the character’s movements, and employing a Reinforcement Learning (RL) technique to select among three, defined control strategies when the character is being pushed or perturbed by the outside world. The character was able to learn simple control strategies for staying upright when being pushed around by an external force by intelligently selecting from three, simple predefined controllers – a “Walk-in-Place” controller, a “Walk Forwards Controller” and a “Walk Backwards” controller.